Sparse nonlinear discriminants

This thesis considers training algorithms for machine learning and their applications for classification, regression and automatic speech recognition. Particularly, supervised learning, which is also called learning from samples, is considered. Starting with a short introduction into statistical learning theory it is shown, that supervised learning can be formulated as a function estimation problem where the class of linear functions turns out to be an appropriate choice for obtaining solutions with high generalization ability. The performance of linear functions may then be enhanced further by the use of the so-called kernel functions, which allow effectively to transform certain algorithms into nonlinear spaces. An important advantage of kernel functions is that the estimated functions are still linear in the new nonlinear space such that the theoretical benefits of linear functions regarding the generalization ability are preserved. The discriminant approach turns out to be appropriate for being transformed into nonlinear spaces using kernel functions. We propose some order-recursive algorithms which allow to estimate a nonlinear kernel-induced version of the discriminant with a reasonable cost. These algorithms are based on two facts. First, the discriminant approach is equivalent to a certain least-squares regression. Second, the solution can be sparsified in the sense that only a small fraction of the training points is